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ABSTRACT 

This report describes the formative evaluation of 
Project CHILD (Computers Helping Instruction and Leeirning 
Development), a project that: (1) integrates computer technology into 
the curriculum eureas of reading, language arts, and mathematics, and 
(2) includes primary (K-2) and intermediate grade (3-5) 
learning-center clusters taught by collaborative teams of three 
teachers specializing in different subject areas. A description of 
the two program sites — Volusia County cmd Okaloosa County school 
districts in Florida — is followed by em outline of the essenticQ 
components of the project, its goals eind objectives, and its 
philosophical and theoretical foundations, highlighting the project's 
focus on opportunities for active leeirning, shared responsibility and 
learner control, cooperation, and fair competition. The four phases 
of the comprehensive evaluation plan — development, initial 
implementation or formative evaluation, continued implementation and 
summative evaluation, and dissemination — are described. The results 
presented are based on: (1) classroom observations and teacher 
meetings; (2) interviews involving administrators and teachers; (3) 
teachers' journals; (4) the nominal group technique (conducted during 
teacher retreats); (5) surveys of teachers, principals, and parents; 
(6) parent feedback (collected during meetings with parents); and (7) 
student performance analysis. Findings are summarized and 11 issues 
concerning components of the project are addressed. The report 
concludes with recommendations concerning teacher journals, the use 
of teacher aides and other volunteers, planning time and time 
management, availability of materials and supplies, student work 
station utilization, reporting to parents, and program continuation. 
(18 references) (GL) 
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Integrtting Computers into the Curriculum: 
A Formttive EvtluatioB 



The purpose of this paper is to report on a project which is successfully integrating 
daily use of the computer with typical elemeniary (K-5) curriculum materials. Project CHILD 
(Ccmputers iielping Instruction and Learning development) has been fully implemented and 
the formative evaluation of the project completed. This paper describes the formative 
evaluation of the Project CHILD system. 

Introduction 

Project CHILD is a research and development project designed to create a model 
educational program to move Florida^s elementary schools into the 21st century. It is based 
upon an interactive approach to learning that integrates computer technology into the 
curriculum areas of reading, language arts, and mathematics. 
Progrjim Sites 

Project CHILD is implemented in two school districts in Florida. One school in Volusia 
County (Northeast Flo/ida) is implementing the project with IBM hardware and software. 
Apple hardware and software is being used at the other site in Okaloosa County (Northwest 
Florida). 

The two project schools which are included are very different in composition and 
therefore provide a good contrast for the implementation of the project with different 
populations. The Volusia County school has a larger proportion of black students (55%) than 
the Okaloosa County school (5%). In terms of student achievement, the Volusia County school 
attains scores on national achievement tests in the average range, however, the Okaloosa 
County school attains high achievement test scores (above the 80th percentile). These two sites 
are representative of a good number of Florida schools and provide a good inferential base for 
the evaluation of the project. Both schools have highly mobile student populations in over 
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populated schools; some classrooms at both schools arc housed in portables. Both schools also 
have ESE students and one school has an entire wing of ESE classes. 



Project Child description 

Project CHILD is based upon a restructured elementary school environment which 
includes primary and intermediate grade learning-center clusters. Teachers in each cluster 
specialize in a subject area and work as part of a collaborative team of three teachers. They 
teach the same students (K-2 or 3-5) in their developmental cluster for three years. Learning 
stations within the classroom provide a variety of learning activities. The computer station 
is where three to six computers are used in each subject on a daily basis. Teachers are trained 
to use computer software as an integrated learning tool rather than a supplement. Twenty 
essential components make Project Child a unique and innovative computer-integrated learning 
system for the elementary schooL 

Essential Componeots 

1. Primary and intermediate grades form classroom dusters. The primary cluster 
includes kindergarten, first and second grades, and grades three, four and five 
form the intermediate cluster. 

2. Teachers are trained as content specialists. Teachers receive special training 
m three content specialties: reading, language arts, or mathematics. 

3. Teachers work in cooperative teams. Teachers within each cluster work 
cooperatively via weekly team meetings, structured observations of each other, 
joint planning and unit culminating ar ities. 

4. Teachers observe students in other classrooms. Once during each six-week unit, 
teachers spend thirty minutes observing students in the two other classrooms in 
their cluster. 

5. Students work with the same teacher team for three years. Students work with 
the same subject area teacher for three years but switch home base teacher for 
each grade level change. Teachers get to know the students and their individual 
learning styles which provides the students more continuity in their learning 
activities from year-to-year. 

6* Teachers use Learning Activities Guides (LAGS) to plan lessons. The LAGS 
are teacher planning tools for each content .specialty. The LAGS are organized 
^nto six six-week units with language arts and reading organized around the 
same themes. The mathematics LAGS are organized into topical chapters for 
greater flexibility in sequencing. Teachers may choose to follow the suggested 
Project CHILD unit sequence or adapt the activities to their text book sequence. 



Statloo activities are clearly defined and appropriate to students* abilities and 
needs. The classroom is set up to accommodate the learning stations described 
in the LAGS. Teachers place Task Cards at the stations to define student 
learning activities. A station can be a designated area with extra tables and 
chairs reserved for station use only, a space on the floor defined by rugs or 
masking tape, or simply a storage shelf from which students retrieve materials 
to take to their desks. 

Teachers use a Dally Station Assignment Sheet (DSAS). The DSAS lets students 
know at which station to begin working each day. 

The classroom ambience Is supportive, equitable, and risk free. Teachers create 
a learning climate which sets high expectations, supports experimentation, is 
equitable for all students and incorporates frequent praise and encouragement. 

Students are trained In management techniques. Students receive a ten-day 
orientation, during which they learn how to use and care for materials and 
equipment responsibly, how to stay on-task while working independently, how 
to move efficiently to various learning stations, and how to use the Passport. 

Students set and assess unit goals. Students are guided by their teachers to set 
reasonable goals at the beginning of each six-week unit and assess their 
achievement at the end of each unit. This information is recorded in their 
Passports. 

Students use Passports each day. Passports are used to record information 
about student goals, opinions, and accomplishments, and to guide their 
movement among the various learning stations each week. 

Students work at a variety of learning stations. The learning stations provide 
a variety of activities designed to accommodate all learning modalities. 
Students work independently at learning stations when they arc not working 
with the teacher. 

Students have frequent and equitable access to computers. Each classroom is 
equipped with a minimum of three computers. Students work at the Computer 
Station at least three times per week in each subject area. 

Students exercise control over materials and equipment. Students have access 
to and control over all materials and equipment at designated learning stations. 
They are trained to use equipment carefully and store materials properly. 

Students work with partners in cooperative teams. At the Computer Station, 
student pairs follow specific strategies to work as cooperative teams (except 
when word processing). They receive recognition and rewards to foster 
cooperation. 

Students know where to get help. The strategies for getting help when the 
teacher is unavailable are clearly communicated to students and are also 
identified on the Task Cards found at each of the learning stations. 

Parents provide Input on a regular basis. At the end of each unit, parents review 
their child^s Passport and write in their own comments. In addition to a parent 
orientation meeting at the beginning of the school year, there is frequent 
communication and parents are encouraged to participate as co*iearners with 
their child and as volunteers in the classroom. 
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Jif*iT*'A % P'"^'.**** >«*dershlp and support. The principal leads a Project 
CHILD Staff meeting once during each six-week unit, visits the classrooms 
frequently and is well-versed in the Project CHILD methods, goals and 
objectives. 

An aide assists primary-grade students at the Computer Station. An aide 
(paraprofessional or volunteer) is available to assist primary-grade students 
each day at the Computer Station in the kindergarten, first and second grade 
classrooms. 



Goals and Objectives 

The ultimate goal for Project CHILD is quite ambitious: to enable more students to 
become successf ul learners. Through Project CHILD, more students who complete elementary 
school can become competent in the foundation skills of reading, language arts, and 
mathematics, and in critical-thinking and problem-solving skills. Project CHILD seeks to help 
children develop a successful foundation for learning through a variety of strategics. Each 
of these strategies constitutes an objective area of Project CHILD and is described below. 

Restructuring the School. Grades K-5 are clustered as primary teams (K-2) 
and intermediate teams (3-5). Classrooms serve as subject-area learning centers 
lor part of each day. The classroom learning centers each contain activity 
stations for hands-on, paper-pencil, textbook, and computer activities. Resources 
(software, textbooks, manipulatives, etc.) are consolidated into the readinf? 
mathematics, or language arts classrooms. 

Restructuring the Curriculum. The standard curriculum has been adapted to 
acconimodate a developmental approach to learning. Learning activities guides 
have been developed which define six units of work. The units have included 
enhanced skills such as critical thinking, problem solving, keyboarding, and 
computer literacy. Specific skills are integrated into the units and aligned with 
Skills covered m Florida's performance standards. Specific computer software 
lessons have been integrated into the units. For three-year spans, students are 
guided through the units by a team of three teachers. Thus students will have 
curriculum specialists who understand their needs and their long-term 
devclopraental progress in the primary grades (K-2) and then in the intermediate 
grades (3-5). 

Empowering Teachers. A teacher's manual and inservice training program have 
been developed to assist teachers in learning new roles and responsibilities. In 
rroject CHILD, the teacher becomes a modern executive, working as part of a 
collaborative team. The teacher becomes a subject specialist, adopting new roles 
as guide and facilitator. By specializing, the teacher can be trained to use the 
computer as an effective learning tool in each discipline. At the same time, by 
working with each child over three years within a developmental setting, the 
teacher retains an important role as nurturer, continuing to meet the critical 
srnotional needs of the young child. 



Empowering Students. Student guides, call Passports, have been developed 
which lead students to set goals, develop responsible work habits, and make 
intelligent choices. Project CHILD students work in cooperative teams and 
learn strategies for helping each other. 

Estftbllshing Healthy Working Environments. A Project CHILD classroom is 
success oriented and fosters a risk*f ree climate in which students are positively 
recognized for effort as well as for accomplishment. Classroom management 
materials (daily station assignment sheets, task cards, award certificates) have 
been developed to establish an orderly.and organized working environment. 

Increasing Motivation and Time*on*Task. Project CHILD students have daily 
access to a variety of learning activities to meet their diverse learning styles 
and attention spans. Access to computers provides the flexibility of guided 
discovery and guided practice, freeing the teacher to have more time to meet 
individual needs. Children can also move about and >york cooperatively in a 
calm atmosphere that promotes the feeling that learning and school are fun. 

Building Bridgeis Beyond the Classroom. Parents arc important partners in tlie 
learning process. A parent orientation has been developed to inform parents 
about the different yet enhanced learninjj opportunities in Project CHILD. At 
the end of each unit, children take home their Passports to review with their 
parents the documentation of their effort. The principal serves as liaison 
between the school and the community as well as instructional leader for the 
CHILD teachers. Volunteers from the community are encouraged to assist 
teachers whenever possible. 

Meeting the Future Needs of Florida's Schools. Project CHILD is designed as 
a cost-effective method of delivering instruction. It has been designed to 
consolidate and effectively utilize existing resources such as textbooks, 
manipulatives, computers, and software. It accommodates a phased-in strategy 
for the purchase of new equipment and software. And the CHILD model has 
potential for accommodating larger class sizes through effective use of 
technology and paraprof essional support. 



Foundations of Project CHILD 

The Project CHILD model is based on sound philosophical and theoretical foundations, 
combining well established approaches to human learning theory, motivational theory, 
behavior modification theory, learning modality theory, and developmental theory into a 
working modeL The project was developed based on operating conditions which exemplify 
active learning, shared responsibility, cooperation, high expectations, and balanced 
curriculum, activities and materials. 
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Philosophical Foundations 



Several philosophical beliefs about education provide the basis for Project CHILD. 
A good education is one where there is involvement, relevance, and thinking (Glasser, 1975). 
Students nr»ust experience repeated success. The following conditions must be present for this 
to happen: 

Active Learning. Students must be actively involved in the learning p-'^cess. 
Frontal teaching" (the teacher imparting knowledge from the front of the 
room) must be balanced with participation of the students in individual learning 
tasks and in small learning teams. Instruction should draw on the experiences 
of the students. 

Shared Responsibility. Teachers mu^t share responsibility with the students 
by giving them opportunities to make choices and decisions affecting their 
learning. Teachers must share control of classroom management by involving 
students in cooperative team meetings that will guide the students toward self- 
discipline. 

Cooperation and High Expectations. The classroom environment must reflect 
a spirit of cooperation and high expectations. Comp?;tition must be fair, giving 
everyone an equal opportunity to succeed. Process becomes as important as 
product; the process of learning and improving must be valued as much as an 
excellent product. 

Balanced Curriculum, Activities, and Materials. The curriculum must be 
balanced with a diversity of subjects, such as reading and language arts, 
mathematics, science, social studies, art, music, foreign language, and physical 
education. Subjects should be integrated around holistic themes as much as 
possible. A variety of learning activities arid materials must also reflect a 
diversity of learning styles and interests. 



Theoretical Foundations 

The structure and content of the Project CHILD model and materials incorporate the 
theoretical principles listed below. 

Human Dynamics Theory. Students wili make an effort to work harder in a 
classroom where their basic needs are being met (Glasser, 1986). These basic 
human needs include belonging, freedom, fun, and power. Students need to 
feel secure in each of these areas in order to maximize their learning. 

Motivation Theory. The effort students will expend on a task is determined 
by the degree to which they expect to be successful and expect the task to meet 
their needs. Both factors must be present. Students will invest no effort on a 
task they perceive either as having no value to them or as being so difficult thac 
they have no expectation of success (Feather, 1982). 

Behavior Modification Theory. Posi'tive reinforcement (rewards can shape 
students' behavior. To be effective, rewards should be immediate and frequent. 
Catching students being good" will help them internalize desired behavior and 
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lead to a decreasing need for external rewards, such as stickers or candy* 
Negative reinforcement (punishments) should reflect logical consequences for 
the bad behavior (Skinner, 1969)« 

Learning Modality Theory. Students learn in different ways and exhibit 
different talents. Classroom materials and learning activities need to 
accommodate these differences so that all students can experience success* 

There are at least four basic learning styles: concrete experience (touching), 
reflective observation (watching), abstract conceptualization (thinking), and 
active experimentation (doing). Some students may rely on one style, while 
other students may employ several or all (IColb, 1983)« 

Hemispheric dominance in the brain may determine whether a student learns 
better through sequential patterns using auditory and visual stimuli (left brain) 
or through global and intuitive patterns using tactile stimuli (right brain). Many 
students may use both spheres effectively, while others show dominance in one 
sphere or the other (Bogen, 1969). 

Students* talents are reflected by at least seven forms of intelligence: linguistic 
(writing), musical, logical (mathematics), spatial (art), bodily kinesthetic 
(athletics), interpersonal (sensitivity to others), and intrapersonal (sensitivity to 
one's own feelings). Students may show strength in several of these areas 
(Gardner, 1983). 

Developmental Theory. As they grow, children pass through stages of 
development, both intellectually and socially. Effective instruction must match 
the child's level of cognitive development, moving from concrete operations to 
abstract concepts (Piaget, 1950). 

A child's emotional development moves from dependence toward independence. 
The learning environment and materials must be structured to match the child's 
social development (Erikson, 1963). 

Children seek order and control. The classroom and its materials must be clearly 
structured and organized to meet this need (Montessori, 1964). 



Project CHILD'S goal is to make more students successful learners. The Project CHILD 
model is based upon the assumption that today's schools must be restructured to meet the 
educational and technological needs of the future. This restructuring must include 
opportunities for active learning, shared responsibility and learner control, cooperation, and 
fair competition* 

The philosophy of Project CHILD calls for variety and balance in the curriculum and 
in learning activities, with emphasis on both process and product. Students need ample 
opportunities to experience success regardless of their learning style and aptitudes. Project 
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CHILD is further grounded in theories that seek to explain motivation, behavior, learning, 
and child development. (Butzin, 1989) 

Comprehensive Evaluation Plan 

Project CHILD has been designed and is being impltmented in four phases. Phase I is 
the development phase and was completed in June, 1988. Phase II, the initial implementation 
or formative evaluation phase, was completed at the end of school in June, 1989. Phase III is 
the continued implementation and summative evaluation phase of the project scheduled for 
the school year 1989-90. The final phase. Phase IV, will be the dissemination of Project 
CHILD materials. 

The focus of the evaluation for Project CHILD parallels the developmental phases of 
the project. During the first phase, evaluation activities were formative, based on a content 
review of project materials. In house reviews consisted of ongoing critiques of all project 
components. Project developers, content experts and staff worked together to review and 
revise the project materials. 

The evaluation focus was on the success of project implementation and on the need 
for revisions to the project materials. Revisions, were based on both the review of content 
materials, and the implementation data from the use of the materials in schools. This report 
presents the evaluation results for the implementation of Phase II of Project Child. This 
formative evaluation or implementation phase was completed with the completion of the 
school year 1988-89. 

Phase III will be implemented in 1989-90 when the effectiveness of the project will 
be evaluated using summative 'valuation methodology. Activities which ^re planned include 
analyses of criterion-referenced and norm-referenced test data, analyses of writing samples, 
interviews with students and parents, analyses of attendance and discipline data, structured 
classroom observations by principals and evaluators, analyses of timc-on-task data collected 
via observations, analyses of the School Attitude Measure, analyses of teacher surveys, 
summaries of Teachers' Journals, analyses of two sets of student passports, and cost 
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effectiveness and policy analyses. A detailed description of the evaluation design is provided 
in the August 1989 proposal for Summative Evaluation (Bergquist and Orr, 1989). 

Formative Evaluation CompGnents 

Formative evaluation activities for Phase II were designed to collect information 
related to four major parts of Project CHILD: LAGs (Learning Activities Guides), Teacher 
Manuals, Student Passports, and inservice training. These essential materials and activities 
were evaluated via the systematic collection of information about them. The evaluation 
activities which were conducted included the following: observations, iaterviews, review of 
Teachers' Journals, two status survey^:, two problem surveys, and a survey of parents. 
Although each activity is different from the others, the significant assets or problems of 
Project CHILD consistently emerged with this multi-strategy approach. 

Evaluation Reports 

Reports on the formative evaluation activities for Phase II were made throughout the 
year. The Summer Inservice was evaluated and a report on the theory and methods of the 
inservice was produced in October 1988 (Junes, 1988). Monthly status reports for November 
1988, and December 1988 (Jones, 1988). In February 1989, a status report on evaluation was 
prepared which included the results of the first status and problem survey (Bergquist and Orr, 
1989). An additional status report on evaluation was prepared and presented in May 1989 
(Bergquist and Orr, 1989). The May 1989 status report included the results of the final status 
and problem survey, summaries of the obs vations, and Teacher's Journals, and some 
preliminary conclusions and recommendations. The final report on the Formative Evaluation 
of Phase 11 of Project CHILD (Bergquist and Orr, 1989) was available in July 1989 and briefly 
summarized each of the results presented in earlier reports. The Final Report also included 
summaries of the survey of parents and the student performance data which had not been 
reported in earlier reports. 
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Classroom Observations and Teacher Meetings 

Bi-wcckly observations by project staff were conducted throughout the year. 
Systematic observations were made and discussed by all project staff, developers and 
evaluators, as a team. Meetings with teachers were held during each visit for further 
clarification and discussion of issues. 

Observations were completed at both pilot school sites. Project staff alternated sites 
and combinations of staff in order to avoid specific observer/site biases. Observations 
indicated that most of the project objectives have been implemented. 

Interviews 

Informal interviews were conducted by project developmental staff and evaluators 
during or after the site observations. These discussions involved school administrators and 
district level staff, as well as teachers. Additionally, follow-up interviews were sometimes 
conducted to clarify field notes or comments made during the observations. Telephone 
interviews were also used to supplement the observations of project staff or to clarify a 
specific concern related by the teachers. 

Teachers' Journals 

Participating teachers documented their daily observations in individual project 
journals. Teachers recorded their observations and impressions regarding the LAGs and 
Passports, the Teacher Manual, time management requirements, and the management of 
student progress. Journals were submitted to the evaluators at the end of each unit. Teachers 
were encouraged to provide candid comments on the project materials, and only the evaluator 
for the project reviewed the individual comments as submitted. These comments were typed 
and organized into another form before they were reviewed by the project development staff. 

Teacher Retreats 

Approximately six weeks after implementation began, all teachers involved in Project 
CHILD spent one day in Tallahassee with Project Staff. Using the nominal group technique 
this time was spent identifying strengths and weaknesses of the components. This valuable 
approach of providing feedback and evaluating project materials was repeated in May. 
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Ssir7cys 

Surveys of t<i^ -:hers, principals and parents were conducted during Phase II. Teachers 
were surveyed twice using the same two instruments each time. The status survey was used 
to address the degree of implementation of project components and the satisfaction teachers 
have with each component. The problem survey asked the teachers to respond to the 
components of the project regarding the degree to which each has presented a problem during 
the implementation. 

The first survey of teachers was completed at the end of unit two in November 1988. 
A second teacher survey was completed in April .^89 so that additional evaluative 
information would be available prior to the completion of Phase II. 

In addition to the surveys of teachers, the principals of the pilot schools were also 
asked to respond to several open-ended questions regarding the implementation of Project 
CHILD. 

Parents were also surveyed by telephone in May 1989. Parents were asked to describe 
the attitudes and reactions of their students to Project CHILD. Parents were also asked to 
respond to questions to provide more information about the students. The ability of parents 
to adjust to a new method of instruction for their children is an important aspect of the 
future implementation of Project CHILD. 

Parent Feedback 

Parent meetings with Project staff have been held throughout the implementation of 
Project CHILD as the need has been indicated. This has proved to be an important aspect of 
the successful implementation at each school site. Parents are informed about the components 
of the project and provided an opportunity to ask questions of and provide feedback to 
project staff. 
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Student Performance Analysis 

As a preliminary indicator of how students in Project CHILD classes perform on 
standardized achievement tests, results for both project schools were collected for 1987-88 
and for 1988-89. The Project CHILD classroom scores were compared to grade level averages 
in both years. These data are only an early benchmark to determine if student performance 
appears to be equivalent to that of other students in the same grade and school. This is 
important to examine during the early implementation of the project but cannot be judged to 
be a true indication of the effectiveness of Project CHILD. 

Formative Evaluation Results 

The results and recommendations presented in this report are based on the 
implementation of Project CHILD during 1988-89. Five evaluation methods were used to 
collect the data for this report: surveys of teachers, surveys of parents, student performance 
data, observations in classrooms and analyses of Teacher Journals. Findings are presented 
in this report for the purpose of revising and improving the project. 

Observations 

Observations have indicated that most of the project components have been 
implemented. Very positive interactions were noted between teachers and students. 
Considerable cooperation and "joint learning" were noted as students worked together at the 
computer stations. The noise level was comparatively low and, for the most part, the noise 
that was identified stemmed from cooperative learning, not off -task or disruptive behaviors. 
Learning activities at stations all appeared to be relevant to the instructional objectives for 
the day and engaged the students in productive learning activities. 

Several program concerns were identified during the observations by evaluators and 
staff. Passports appeared to be used in a variable manner by students. In many cases they 
were only "clutched", not referred to, as students moved from station to station. Although 
some students marked in the Passports when at ihe stations, most of the marking appeared to 
be at the prompting of the aide or the teacher at the end of the class pe.iod. 
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The amount of teacher time needed to plan for station activit/.es and create the 
supporting materials for them was questioned as a logistical factor associated with the 
program. 

Some teachers were observed conducting extensive whole group activities. This raises 
some questions concerning the degree of use for stations. There were instances noted, as well, 
where use of stations was made a contingency for some other desired classroom behavior 
instead of treating stations as an integral component of the classroom. Some instances were 
noted as well of station activities that had no feedback mechanism to let students know if the 
answers were right or wrong. 

There appeared to be some problems with the instructional purpose and quality of 
some of the software in use when students were observed working on the computer:^. The 
degree of on-task behavior of a few students at the computers and at other stations was also 
identified as an area of concern. Although this was not a general classroom problem, certain 
children were observed repeatedly off-task. 

Teachers' Journals 

All teachers participating in the project were asked to document observations in 
individual project joirnals. Teacher^s recorded their oDservations and impressions daily (by 
grade level) regarding the LAGs and Passports, the Teacher Manual, time management 
requirements, and the class management of student progress. Journals were submitted to 
project evaluators at the end of each unit. To encourage candid comments by the teachers, 
only the evaluators for the project have reviewed the individual teacher comments as 
submitted. The comments from all teachers were typed and organized into another form 
before they were reviewed by the project development staff. 

Comments related to all aspects of Project CHILD have been found in the notes made 
in the Teachers* Journals. A comprehensive content analysis of these comments has been 
completed and is found in other more detailed reports. Overall, most of the teachers rated 
their classroom experiences on a daily basis with Project Chilo as excellent or good. There 

" le 



were some ratings that were less positive but did not reflect a consistent trend across teachers 
snd schools. 

Parent Survey 

Interviews with parents were conducted during Phase II to obtain thvir reaction to 
the involvement of their chiidren in Project CHILD. The interviews with parents were 
conducted in the first week in May. A randomly selected group of parents from each grade 
level and school were asked to respond to a series of 6 to 8 questions regarding Project CHILD 
(6 for kindergarten and 8 for all others). 

Samples were selected from alphabetized lists of students with a total of 28 selected 
from Westside, and 25 selected from Valpariso. Home and work telephone numbers were 
obtained from the school when these were available and three attempts were made to contact 
the parents, each at different times of day. The interviewer was able to contact and obtain 
responses from 15 of the 28 Westside parents and 13 of the 23 Valpariso parents, representing 
more than iO% of the Project CHILD students. 

The results and conclusions regarding the parent responses are summarized below. 

1. Parents seemed to know quite a lot about Project CHILD. Only 3 of 24 parents 
indicated that they did not know anything about Project CHILD. As expected, 
the most frequently named difference was the use of computers, but also 
significant, as perceived by parents, was the use of three teachers and switching 
classes. " 

2. Parents overwhelmingly indicate that their children have a positive attitude 
toward school. Although no comparisons with other students were made this 
year, this by itself seems significant in terms of how students have responded 
to the project. The number of parents who indicated that they were satisfied 
with the program was equal to the number of students who like school a lot 
The students have been indicating their enjoyment of Project CHILD to project 
staff and the evaluators during their on-site visits and in their comments 
recorded m Passports. 

3. Six parents who responded to the survey questions indicated that their child's 
attitude toward school was worse this year but 18 indicated no change or a 
better attitude. This is not a negative statement about children's attitudes 
because m question 2 parents have indicated that only 3 children do not like 
It, and 3 think it is just o.k. We conclude that although some parents have 
indicated that their children's attitude toward school is worse, it is still 
somewhat positive. 

4. Overall, parents were satisfied with the program (19 of 28 responded yes). The 
responses of patents have pointed out what they do and do not like about the 
program. Although parents are generally satisfied with the program, it is 
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important to review the comments of the parents who were not satisfied for 
herein are the areas which should be targeted for improvement. It seems that 
while a few parents were concerned about contend matters, most of the 
comments are related to the individual attention that was received by their 
children. While this may be an issue with which schools in general arc dealing, 
it is an important one for a new program to address. Project staff should work 
with teachers on how to communicate to parents that the needs of all individuals 
are being addressed. Clearly, some of the parents felt that this was 
accomplished effectively. 

The schools invohed in the 1988-89, were very different in terms of the 
assistance of an aide with the computer in the classroom. One school, Westside, 
had aides to assist with the computers on an ongoing basis, but the other school 
had to rely on parent volunteers. This factor may have contributed to the fact 
that parents felt teachers need to spend more individual time with their child 
and were unable to do so. 

The responses of the parents about what the children liked best arc no real 
surprise. Observations and informal talks with students had already identified 
that the students and parents both were excited about the addition of computers 
to the classroom. The variety of other aspects that were mentioned was 
encouraging and indicates that Project CHILD has met a variety of interests. 

Although most parents are satisfied with Project CHILD, several suggestions 
for improvements have been made by parents. The comments of parents can 
be grouped around two issues: content emphasis and organizational strategies. 
Clearly the parents who have responded to cither of these issues feel that the 
attention their child was receiving was not sufficient. Again we recommend 
that project staff work with teachers on how to communicate to parents that 
the needs of all individuals are being addressed. 

Most parents were satisfied with the program; 23 of 28, with only 3 parents 
indicating that they were not satisfied. While this does not seem like many 
parents, if these results were generalized to the total population of parents, this 
would indicated that approximately 10% of the parents involved would not be 
satisfied with Project CHILD. From other questions we have learned that an 
area of dissatisfaction is that parents do not feel their children are receiving the 
individual attention they need. This may be an ongoing problem for any 
classroom with 28 students and 1 teacher, however, further research would be 
needed to determine this, this area should be improved, if possible. The 
parents' attitude may change as they receive more feedback on the performance 
of their students on tests as compared to past years and compared to other 
students in the school. 

Responses to recommending the program to others are very positive. It is 
interesting to note that some parents have expressed the belief that this program 
is not good for slow learners or low ability students. There is presently no 
evidence that this is true or not true but it is something the project staff and 
evaluation team should be investigating during Phase IIL 
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Teacher Surveys 

On two different occasions surveys were administered to all of the teachers (12) 
participating in the project. Each teacher answered questions regarding the status of the 
implementation of Project CKILj3 and problems which may have occurred during the 
implementation. The results are only briefly summarized below because they have been 
reported in more detail in earlier reports. The detailed resuUs of Survey #1 arc reported in 
the February 1989 Status Report on Evaluation snd the detailed results of Survey #2 are 
appended to the Status Report on Evaluation dated May 15, 1989. 

Teachers were asked first to rate each status survey item in terms of the degree to 
which the item as been implemented to date. A 6-point scale was used for degree of 
implementation and ranged from a low anchor point of **! = No implementation planned** to 
a high point of **6 « Complete implementation.** Another scale indicated their present degree 
of saticfuction, and ratings ranged from a low of "1 = Extremely unsatisfactory - delete** to a 
high of "4 a Highly satisfactory.** 

The problem survey was designed to assess the level at which various problems exist 
in relation to the components of the CHILD model. Teachers were asked to rate each survey 
item in terms of the severity of the problem. A 4-point scale was used and ranged f '•om a low 
anchor point of **1 » Not a problem** to a high rating point of **4 = A very serious problem.** 
At the conclusion of the survey, teachers were asked to go back and circle the ten areas of 
problems which most seriously inhibited the success of the project. 

Status Survey #1 Results. The results of the status survey indicated very high levels 
of implementation for most of the project components. At least two of the project components 
were never fully implemented: classroom aides at Valpariso and the use of award certificates 
at both schools. Additionally, most of the project components were given high satisfaction 
ratings. The most favorable rating combination for the project is a high level of 
implementation and a high level of satisfaction. The following areas were given this rating. 

* Use of the goals page. 

* Use of the station visitation chart. 



16 



The computers. 

The computer software recommended in the LAG. 
Partners as cooperative learning teams at the computer. 
Strategies for students getting help other than from the teacher. 
Supportive risk free environment. 



One area which was rated with a low level of implementation but with a high level 
of satisfaction was the use of the award certificates. Based on these results, more utilization 
of this component was encouraged. 

Problem Survey #1 Results* When the teachers were asked to identify the 10 most 
serious problems for implementation of Project CHILD components only 3 were identified 
by 6 (half) or more of the 12 teachers. These problems are listed below with the numbers in 
parentheses indicating the number of teachers who identified this as a problem. These three 
problems can be placed into two categories: materials/activities availability and time required 
to implement a new program. Both of these issues are not unusual during the first year of 
implementihg any new program. These issues will be monitored closely in Phase lU. 



Finding sufficient materials to make stations activities. (7) 
The amount of time required to make activities. (7) 
The amount of time required for lesson planning. (6) 



When problem areas are rated and reported, there is a tendency to overlook the positive 
aspects of a program. Much of the feedback received to date indicate that there are many 
positive aspects of Project CHILD, including the excitement of the students about learning, 
their ability to assist each other and be self paced, the successes of children previously having 
difficulties and many more. In fact, survey results indicate that for the majority of the 
components there are not problems. At least half of the teachers have indicated that 17 of the 
59 items listed on the problem survey are '^not a problem** and on another 26 of the 59 items 
that there is "not a serious problem". 

Status Survey #2 Results. The results of the status survey indicated high levels of 
implementation for almost all of the project components and levels that were even higher 
than reported in November, indicating almost full implementation of the program. Of the 
27 items, only one item was not rated by a majority of the teachers as implemented. This 
item related to award certificates. 



17 



20 



In addition, most of the project components were given high satisfaction ratings by 
the majority of project teachers. The most favorable rating combination for the project is a 
high level of implementation and a high level of satisfaction. More than 70% of the items 
(19 of 27) were rated by a majority of the teachers as highly implemented with a high degree 
of satisfaction. The areas of high implementation and high satisfaction are as follows: 

* Teacher^s Manual as a reference for classroom management 
Two-week Student Orientation as a daily guide for training students 

* Station Task Cards. 

* Unit overview 

* Unit:planning guides 

* Teaching tips 

* Recommended stations 

* Suggested station activities 

* Materials and' resource list 

* Station pages 

* Evaluation pages 

* Station visitation charts. 

* Hands on materials 

* Use of computers 

* Recommended computer software 

* Partners as cooperative learning teams at the computer 
Strategies for students getting help other than from the teacher 
Weekly cluster team meetings 

* Supportive risk-free environment 

This list includes twice as many areas as were identified in the first Status Survey. 
No areas were rated with a low level of implementation but with a high level of satisfaction. 
The low implementation and low satisfaction results have been integrated into the discussion 
of issues in a later section of this report. 

Problem Survey #2 Results* Of the 58 total items on the survey, 35 fitems (60%) were 
rated as "not a serious problem" by a majority of the teachers. Nine of the 58 items (16%) 
were rated by a majority of the teachers as "a serious problem". These are listed below. 

* Accurate recording of students practice activities in the Passport 
Students using Passports to make appropriate station selections 
Tracking student progress at the computers 

* Finding sufficient materials to make station activities 
Time required to make activities 

* Preparing activities to meet the needs of varied student abilities 
Tracking student progress at the stations 

* Time to work/plan with other Cluster Team members 
Providing sufficient help for lower ability students 
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A total of 25 of the S8 items (43%) were rated by two or more teachers as a very serious 

problem. It should be noted that, although an item may be rated by a majority of teachers as 

a non-problem area in general, some teachers may still perceive the area to be a severe 

problem. The items identified as **a very serious problem** by some but not a majority of the 

teachers are listed below. 

^ Keeping students on task without opening all stations (4) 
Time required for lesson planning (5) 

Teachers reported that more time is being spent in station activities and less time is 
spent this year in whole group activities. Teachers a!^o reported referring to the Teacher's 
Manual often or somewhat, although three teachers indicated that they seldom or never refer 
to the Teacher's Manual. Likewise, most teachers indicated that they refer often or somewhat 
to the LAGs, although there was a distinct difference in the responfij by school. Most of 
the teachers review the students Passports somewhat or often. Compared to a more tr?.ditional 
classroom, most of the teachers perceive their students to be learning more (7/12), on task more 
often (8/12), working more independently (10/12), and reading better (10/12). Six of the 
teachers indicated that the number of computers in the classroom is not enough. The number 
of stations was rated as just right by 10 of the 12. teachers. Most of the teachers (8/12) 
indicated that they arc enjoying their jobs more this year. 

Student Performance 

As a preliminary indicator of how students in Project CHILD classes perform on 
standardized achievement tests, results for both project schools were collected for 1987-88 
and for 1988-89. The Project CHILD classroom scores were compared to grade level averages 
in both years. These data are only an early benchmark to determine if student performance 
appears to be equivalent to that of other students in the same grade and school. This is 
important to examine during the early implementation of the project although it cannot be 
judged to be a true indication of the effectiveness of Project CHILD. Test scores for both 
schools, grades 2-5, are presented in Table 1 for Valparisc Elementary School and Table 2 for 
Westside Elementary School. 
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An examination of these data reveal several important differences but also some 
common elements in terms of how the student populations look from year to year and also in 
terms of how Project CHILD classes compare to the present and past year school performances. 



1. 



The schools differ significantly in terms of their level of performances. 
Valpanso students score very high on standardized tests with the National 
f.^'^??'"''" ?he school averages ranging from 72 to 87 for the year presented. 
Very few^schools nationwide score this high. With scores as high as this, a 
ceiling effect may occur and improved performances will be difficult to detect 
o» a «anaara«cd achievement test. Westside on the other hand is a school with 
solidly average student performances. Their National Percentiles of the school 
averages range from 42 to 68 for the two years presented. There is a greater 
possibility m these score ranges for increased performances to be detected on 
a standardized achievement test. 



Table 1 

Valpariso Elementary School 
Comprehensive Tests of Basic Skills 

Test Results in Scale Scores* 



Grad^ 








LaURuaBe Mathematics 


Total 


2 


91 
24 


87- 88 

88- 89* 
CHILD 


Results are 

634 

640 


"632 
640 


'.g from the school. 
648 
663 


638 
648 


3 


129 
93 
22 


87- 88 

88- 89* 
CHILD 


659 
65y 
654 


669 
667 
661 


671 
669 
652 


667 
665 
656 


4 


121 
113 
28 


87- 88 

88- 89* 
CHILD 


713 
707 
712 


701 
685 
706 


693 
691 
707 


702 
698 
710 


5 


112 
99 
24 


87- 8C 

88- 89* 
CHILD 


731 
735 
691 


720 
718 
723 


711 
712 
710 


720 
722 
724 



Results for 1988-89 do not include the Project Child classes which 
are reported separately. 



2. Both schools have performances on the 1988-89 standardized achievement tests 
Which are essentially the same as those for the school in 1987-88. There is one 
exception: 88-89 students in grade four at Westside appear to have performed 
better than Westside fourth graders did in 87-88. 
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Table 2 
Westside Eietnentary School 
Comprehensive Assessment Program • Achievement Series 



Test Results in Scaie Scores* 







Year 


Reading 


Lanciiase 


Math ctn Sit ICC 


Total 


2 


79 


87-88 


411 


414 


443 


434 




68 


88-89 


408 


409 


425 


424 




22 


CHILD 


421 


420 


433 


435 


3 


97 


S7-88 


471 


446 


512 


491 




58 


88-89 


462 


447 


496 


485 




21 


CHILD 


503 


541 


546 


550 


4 


102 


87-88 


509 


469 


537 


513 




50 


87-89 


529 


498 


553 


536 




25 


CHILD 


545 


504 


554 


543 


5 


80 


87-88 


538 


506 


596 


552 




62 


88-89 


537 


495 


575 


541 




26 


CHILD 


530 


502 


581 


541 



Results for 1988-89 do not include the Project Child classes which are reported 
separately. 



3. The schools are somewhat different in the achievement of Project CHILD 
students as s^ompared to the performance of all ether students in the school. 
The evide»»" * of higher performances by Project CHILD classes at Westside 
which u ..ut observed at Valpariso is most likely due to the ceiling effects of 
the tests used at Valpariso. Caution should be used in interpreting these results 
as preliminary indicators only. The Phase III evaluation design will address 
program effectiveness both by design and by collecting data which is more 
directly related to Project goals. 

At Valpariso, as expected, most of the Project CHILD classes perform 
approximately the same as their grade level counterparts in 88-89 and in 87-88. 
There are only five cases where performances arc different by 10 or more scale 
score points. Project CHILD classes scored a little better in second grade 
mathematics and in fourth grade language and mathematics. Project CHILD 
classes also scored slightly lower in third grade math and fifth grade reading. 
These fluctuations should be considered normal, and the overall performance 
should be considered to be approximately the same. 

At Westside, the Project CHILD Classes have performances which are mostly 
higher (by more than 10 or more scale score points) than the other students in 
the school, especially at second and third grades. At these grades reading, 
language and mathematics scores are higher for Project CHILD classes than 
for the other students at these grades. Fourth graders in Project CHILD classes 
also perform higher than other students in reading, but essentially the same in 
other subjects. There are no real differences in performances in fifth grade 
Project CHILD classes and all other students in any subject area. Overall, these 
results indicate that Project CHILD classes are performing about the same as 
students in other classes. 
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In summary, preliminary data indicate that students in Project CHILD classes can 
perform as well as students in a traditional classroom environment on standardized 
achievement tests. There is also some indication that they may be able to score slightly higher 
than students in other classroom situations, at least at some grade levels. There is no 
indication that the results reported herein are spuriously high, in fact, they may be 
conservative estimates of performances given the already high performances of Valpariso 
Elementary School students. 



Findings and Issue Summary 

The primary focus of formative evaluation is to improve the program by identifying 
areas in need of revision. This section summarizes the results from on-site observations. 
Teachers' Journals, ar.-. survey results. Issues concerning 1 1 components of Project CHILD 
are addressed. In general, the issues are related to specific components of Project CHILD. 
While the specific issues raised in a formative evaluation may be different for a different 
project, it is typical and desirable to identify areas where improvements in the program can 
be made. The 1 1 issues addressed in this section are not unusual for a formative evaluation 
to uncover. Collectively, all of the data indicate that Phase III of Project CHILD should 
proceed and recommendations for this Phase are made in the Conclusions and 
Recommendations Section. 

Issue 1: Passports 

Problems have been expressed concerning the difficulty of getting students to mark in 
their passports. Overall, however, survey results and Teacher Journals indicate that most 
students are using passports and are recording in them. Observations noted considerable 
prompting by the teacher tc ensure that students recorded at least by the end of the period. 
Accuracy of the recording was one of the most negatively marked items on the survey (8 of 
12 indicating a serious problem). 
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Revisions for 1989-90: Passports and Task Cards have been redesigned, student 
orientation and training procedures enhanced. 
Issue 2: OrlentatioB 

Most teachers indicated that the orientation sections are being implemented and th&t 
they arc satisfied with them. The revisions in the units appear to have satisfied the problems 
previously expressed in this area. The survey to parents has pointed out that an improvement 
could be made in the orientation which is provided to parents. More information provided to 
the parents throughout the year would be helpful as well, especially as individual student 
needs and provisions for these needs can be addressed. 

• Revisions for 1989-90: Th^ Teacher's Manual has been revised. Letters to parents are 
now provided in T.M., LAGs, and Passports. 

Issue 3: Planning Time 

Planning time was mentioned repeatedly in the Teacher Journals (although not at all in 
the Unit 6 journals) as a problem arcs and was rated as a very serious problem by 5 of the 12 
teachers. Teachers documented in their Journals staying until 6 and 7 p,m. and working 
weekends to complete ^he clcan-up from one day and preparation for the next day's activities. 
Time required to make activities was identified as a very serious problem by 6 of the 12 
teachers. The issue planning time as it relates to developing station activities is being 
addressed duri^ig the summer. 

♦ Revisions for 1989-90: LAGs have been consolidated. Daily plans are provided for all 
stations for Unit One. Teachers will compile gtation Activities Rcsou.cp BoQks for use 
in Phase III. Planning Time will continue to be a concern to the evaluators in Phase III 
and will be monitored closely. 

Issue 4: Daily Station Assignment Sheet (DSAS) 

Although preparation of the DSAS was a problem identified in an earlier report, the 
second survey documented that most teachers (8 of 12) do not perceive this area to be a 
problem. Some of the teachers are moving students in groups which reduces the amount of 



23 

26 



preparation time. In unit six some teachers reported letting students choose computer partners 
within their own group. The students responded well to this. 

♦ Revisions for 1989-90: The number of stations and names have been changed. 
Issue 5: Time Managemeot 

Journals for units 1-5 indicated that teachers believe the program does not allow 
sufficient time to complete the tasks that are part of Project CHILD. This issue wasn't even 
mentioned in unit six. Teachers noted that they couldn't cover everything in the time span 
allowed and also stated that there was a lack of time to work and plan with other Cluster 
Team members (7 of 12 indicated this was a serious problem). It was unclear whether it 
disappeared as an issue in unit six or if it was just not noted. 

* Revisions for 1989-90: Principals have adjusted schedules. 
Issue 6: Aides or Volunieers 

Problems with both aides and with the volunteers who help with the computers were 
noted. On the survey, many teachers marked Not Applicable because they are not using aides 
or volunteers, making the majority criterion inappropriate for these items. Journals 
documented repeated instances of the volunteers not coming, new aides not understanding 
about computers, and cither an unwillingness or fear of learning (observations noted this as 
well). At least one teacher cited that this problem still existed during the last six-weeks of 
school. 

♦ Revisions for 19^,9-90: Turnover problem is unresolved. 
Issue 7: Utilization of Computers 

Observations by the evaluators and project staff have documented good time utilization 
of the computers in the second semester of the program. Likewise, use of the computers was 
identified as one component in which teachers were satisfied with the implementation. This 
problem appears to have been solved. 

* Revisions for 1989 90: Additional software acquired to fill curriculum gaps. 
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Issue 8: Task Timing 

The issue of the fixed timing of the computer lessons, reducing the adaptability to meet 
the needs of the students has appeared to be solved and was not mentioned as a problem for 
the later time periods. Teachers were told to adjust the time of the lesson where needed to 
accommodate this concern, and this direction appears to have solved the problem. 

♦ Revisions for 1989*90: Flexible scheduling encouraged. 
Issue 9: LAGs 

After unit three, teachers begin implementing revised LAGs which the project staff 
continued to update. Most teachers are satisfied with this implementation. Some comments 
in the Teacher's Journals were made concerning the quality of the materials. Survey results, 
however, documented 10 of 12 teachers did not consider using the LAGs to be a serious 
problem. 

* Revisions for 1989-90: Final revision and consolidation of units is ongoing. Enhanced 
elements include sections on setting goals, evaluation strategies, providing for diverse 
needs, skills checklists, daily lesson plans for unit one (weekly plans thereafter), 
correlation of station activities with the Station Activities Resource Books, and appendix 
materials for content background information. 

Issue 10: Instructional Time and Mode 

Observations and Teacher Journals again documented that some teachers have continued 
to rely on whole group instruction, rather than moving their students toward independence. 
Responses from teachers, however, document that most teachers spend 50% of the time in small 
group instruction and less than 2S% in whole group instruction. Although the problem was 
reduced as the year progressed, some classes still continue to receive considerable mstruction 
through whole group instruction. 

During unit S, one week was taken up with Spring Break and at least two more weeks 
were devoted primarily to preparing for and taking the achievement tests administered to 
the whole school in each district. During unit 6, some Teacher Journals documented 
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considerable amounts of time away from Project CHILD implementation during this period 
due to testing and end of the year activities. 

♦ Revisions for 1989-90: Time away from instruction due to outside pressures is 
unresolved. 

Issue 11: Materials and Supplies 

A new issue which surfaced in the unit 5 Journals was the concern about running out 
of supplies and materials for the program. It could not be determined whether this is a school- 
wide problem or specific to Project CHILD. Staff should observe what happens in Phase III 
before conducing that this is a problem created by Project CHILD. 

* Revisions i or 1989-90: Being monitored. 

Recommendations 

The recommendations presented in this section are based on all of the findings of the 
Formative Evaluation of Project CHILD. 

1. Teaclier Journals: The format for the Teacher Journals should be revised for next year's 
evaluation using one page per week format with a checlc-list included. 

2. Use of Teacher Aides/Volunteers: At the present time, it appears that the use of 
aides/volunteers in grades K-2 is a critical component for the implementation of Project 
Child. If aides/volunteers cannot be provided in the primary classroom, teachers need 
more training in the management of the computer station without the use of aides or 
volunteers. This factor should be examined carefully through next year's evaluation. 

3. Planning Time and Time Management: The extensive time needed to plan and implement 
the program continues to be a concern. Although some of the time will be reduced 
following revisions to the program this year, this area should reviewed carefully by 
the Project Staff in Phase IIL Systematic methods should be identified for reducing 
planning time and increasing the teacher's facility in the logistics of implementing 
Project CHILD. 
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4* Materials and Supplies: Several teachers noted in their Teacher Journals that materials 
and supplies were very low. The prevalence and severity of this problem should be 
addressed immediately by Project Staff through interviews with teachers and principals. 
Project Staff should seeic to determine if this is a school wide problem typical of the time 
of year or if it is related to the implementation of Project CHILD. 

5. Statios Utilization: There has been some question about the amount the stations are 
utilized and the stations which are utilized the most. This should be monitored in Phase 
III by observers, the teacher journal reports, znd by an analysis of student passport 
information. 

6. Parent Information: As a help to teachers and parents, project staff should work on a 
form for reporting to parents. If possible it should be revised to be more personal. 
Parents could be asked at the beginning of the year what their children's areas of need 
are and the teachers could respond with a year long strategy for meeting their needs. 

7. Program Continuation: Preliminary evidence gathered to date indicates that this program 
can be implemented in typical elementary schools. The program should be implemented 
in 1989-90 with a systematic summative evaluation to determine the impact on students. 

The results and recommendations presented in this report are based on the formative 
evaluation of the implementation year of Project CHILD. Five data collection methods were 
used to compile this report: surveys of teachers, surveys of parents, student performance data, 
observations in classrooms and analyses of Teacher Journals. Findings are presented in this 
report for the purpose of revising and improving project materials. Additional evaluation 
information will continue to be collected during Phase HI of Project CHILD during 1989-90 
in order to address project effectiveness. 
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